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Q ■ Abstract 

, The combination of source coding with decoder side-information (Wyner-Ziv problem) and channel 

coding with encoder side-information (Gel'fand-Pinsker problem) can be optimally solved using the 

separation principle. In this work we show an alternative scheme for the quadratic-Gaussian case, 

which merges source and channel coding. This scheme achieves the optimal performance by a applying 

c/3 i modulo-lattice modulation to the analog source. Thus it saves the complexity of quantization and channel 

decoding, and remains with the task of "shaping" only. Furthermore, for high signal-to-noise ratio (SNR), 

CO ' the scheme approaches the optimal performance using an SNR-independent encoder, thus it is robust to 

> ■ 

If^ ' unknown SNR at the encoder 
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keywords: joint source/channel coding, analog transmission, Wyner-Ziv problem, writing on dirty 
paper, modulo lattice modulation, MMSE estimation, unknown SNR, broadcast channel. 

I. Introduction 

Consider the quadratic-Gaussian joint source/channel coding problem for the Wyner-Ziv (WZ) source 
[1] and Gel'fand-Pinsker channel [2], as depicted in Figure|Ol In the Wyner-Ziv setup, the source is jointly 
distributed with some side information (SI) known at the decoder. In the Gaussian case, the WZ-source 
sequence Sk is given by: 

Sk = Qk + Jk , (1) 

where the unknown source part, Qj^, is Gaussian i.i.d. with variance ctq, while Jj^ is an arbitrary SI 
sequence known at the decoder. In the Gel'fand-Pinsker setup, the channel transition distribution depends 



^Parts of this work were presented at ISIT2006, Seattle, WA, luly 2006. This work was supported by the Israeli Science 
Foundation (ISF) under grant # 1259/07, and by the Advanced Communication Center (ACC). The first author was also supported 
by a fellowship of the Yitzhak and Chaya Weinstein Research Institute for Signal Processing at Tel Aviv University. 
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Figure 0: The Wyner-Ziv / dirty-paper coding problem 



on a state that serves as encoder SI. In the Gaussian case, known as the dirty paper channel (DPC) [3], 
the DPC output, Yk, is given by: 

Yk = Xk + Zk + Ik , (2) 

where is the channel input, the unknown channel noise, Zj., is Gaussian i.i.d. with variance N, while 
/fe is an arbitrary interference, known at the encoder. When referring to and J^, we use the terms 
interference and SI interchangeably, since they may be seen either as external components added to the 
source and to the channel noise, or as known parts of these entities. From here onward we use the bold 
notation to denote iC-dimensional vectors, i.e. 

X = [Xi, ■ ■ ■ , Xk, • • • , Xk] ■ 

The sequences Q, J, Z and I are all mutually independent, hence the chaimel noise Z is independent of 
the chaimel input sequence X. The encoder is some function of the source vector that may depend on 
the chaimel SI vector as well: 

X = /(S,I) , (3) 



and must obey the power constraint 



^E{\\Kf}<P, (4) 



where || • || denotes the Euclidean norm. The decoder is some function of the channel output vector that 
may depend on the source SI vector as well: 

S = 5(Y,J) , (5) 

and the reconstruction quality performance criterion is the mean-squared error (MSE): 

D=^E{\\S-S\\^} . (6) 



2 



The setup of Figure |0] described above is a special case of the joint WZ-source and Gel'fand-Pinsker 
channel setting. Thus, by Merhave and Shamai [4], Shannon's separation principle holds. So a combination 
of optimal source and channel codes can approach the optimum distortion D°p^, satisfying: 



where i?wz(^) is the WZ-source rate-distortion function and Cdpc is the dirty paper channel capacity. 
However, the optimality of "digital" separation-based schemes comes at the price of large delay and 
complexity. Moreover, they suffer from lack of robustness: if the channel signal-to-noise ratio (SNR) 
turns out to be lower than expected, the resulting distortion may be very large, while if the SNR is 
higher than expected, there is no improvement in the distortion [6], [7]. 

In the special case of white Gaussian source and channel without side information (I = J = 0), it is 
well known that analog transmission is optimal [8]. In that case, the encoding and decoding functions 



are mere scalar factors, where /? is a "zoom in" factor chosen to satisfy the channel power constraint 
and a is the channel MMSE (Wiener) coefficient. This scheme achieves the optimal distortion (jT]) while 
having low complexity (two multiplications per sample), zero delay and full robustness: only the receiver 
needs to know the channel SNR, while the transmitter is completely ignorant of that. Such a perfect 
matching of the source to the channel, which allows single-letter coding, only occurs under very special 
conditions [9]. 

In the quadratic-Gaussian setting in the presence of side information, these conditions do not hold [4]. 
It is interesting to note that in this case, i?wz(^) is just the Gaussian rate-distortion function for the 
unknown source part Q [5], while Cqpc is just the AWGN capacity for the channel noise Z [3], i.e. the 
SI components I and J are "eliminated" as would be done had they been known to both the encoder 
and the decoder. We see, then, that this perfect interference cancelation is not achievable by single-letter 
coding. 

In this work we propose a scheme for the joint Wyner-Ziv/dirty-paper problem that takes a middle 
path, i.e., a "semi-analog" solution which partially gains the complexity and robustness advantages of 
analog transmission: It can be made optimal (in the sense of (|7]l) for any fixed SNR, with reduced 
complexity. Moreover, it allows a good compromise between the performance at different SNRs, and 
becomes SNR-independent at the limit of high SNR. 



(7) 



f3Sk 



(8) 
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The scheme we present subtracts the channel interference I at the encoder modulo-lattice, then uses 
again subtraction of the source known part J in conjunction with modulo-lattice arithmetic at the decoder. 
Thus it achieves an equivalent single-letter channel with I = J = 0. Since the processing is applied 
to the analog signal, without using any information-bearing code, we call this approach modulo-lattice 
modulation (MLM). 

Modulo-lattice codes were suggested as a tool for side information source and channel problems; 
see [10], [11], where a lattice is used for shaping of a digital code (which may itself have a lattice 
structure as well, yielding a nested lattice structure). Modulo-lattice transmission of an analog signal in 
the WZ setting was first introduced in [12], in the context of joint source/channel coding with bandwidth 
expansion, i.e. when there are several channel uses per each source sample. Here we generalize and 
formalize this approach, and apply it to SI problems. In a preliminary version of this work [13], we used 
the MLM scheme as a building block in Analog Matching of colored sources to colored channels. Later, 
Wilson et al. [14], [15] used transmission of an analog signal modulo a random code to arrive at similar 
results. Recently, MLM was used in network settings for computation over the Gaussian MAC [16] or 
for coding for the colored Gaussian relay network [17]. 

The rest of the paper is organized as follows: In Section |ll] we bring preliminaries about multi- 
dimensional lattices, and discuss the existence of lattices that are asymptotically suitable for joint WZ/DPC 
coding. In Section |lll] we present the joint WZ/DPC scheme and prove its optimality. In Section |IV] we 
examine the scheme in an unknown SNR setting and show its asymptotic robustness. Finally, Section |V] 
discusses complexity reduction issues. 

II. Background: Good Shaping Lattices for Analog Transmission 

Before we present the scheme, we need some definitions and results concerning multi-dimensional 
lattices. Let A be a K-dunensional lattice, defined by the generator matrix G G R^^^. The lattice 
includes all points {1 = G • i : i G Z^} where Z = {0, ±1, ±2, . . .}. The nearest neighbor quantizer 
associated with A is defined by 

Q(x) = argmin llx — 111 , 
^ leA ' 

where || • || denotes the Euclidian norm and ties are broken in a systematic manner. Let the basic Voronoi 
cell of A be 

Vo = {x : Q(x) = 0} . 
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The second moment of a lattice is given by the variance of a uniform distribution over the basic Voronoi 
cell: 



K 

The modulo-lattice operation is defined by: 



Vo 



X mod A = X — (5(x) . 
By definition, this operation satisfies the "distributive law": 

[x mod A + y] mod A = [x + y] mod A . (10) 
The covering radius of a lattice is given by 

r(A) = max ||x|| . (11) 

xGVo 

For a dither vector d, the dithered modulo-lattice operation is: 

y = [x + d] mod A . 

If the dither vector D is independent of x and uniformly distributed over the basic Voronoi cell Vo, then 
Y = [x + D] mod A is uniformly distributed over Vo as well, and independent of x [18]. Consequently, 
the second moment of Y per element is cr^(A). 

The loss factor L(A,pe) of a lattice w.r.t. Gaussian noise at error probability pe is defined as follows. 
Let Z be Gaussian i.i.d. vector with element variance equal to the lattice second moment (T^(A). Then 

L(A,pe) = mill |z : Pr|-^ ^ Vol < Pel • (12) 



Vi 

For small enough p^. this factor is at least one. By [19, Theorem 5], there exists a sequence of lattices 
which possesses a vanishing loss at the limit of high dimensiorQ, i.e.: 

lim lim L{Ax,Pe) = 1 • (13) 

Moreover, there exists a sequence of such lattices that is also good for covering, i.e. defining: 

r2(A) 

^(*) = TOT • 

where r(A) was defined in (fTTI) . the sequence also satisfiej^: limK^oo L{Ak) = 1. However, for this 
work we need a slightly modified result, which allows to replace the Gaussian noise by a combination 

'These lattices are simultaneously good for source and channel coding; see more on this in Appendix U 
^Note that by definition, L{Ak) > 1 always. 
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of Gaussian and "self-noise" components. To that end, we define for any < a < 1 the a-mixture noise 



as: 



= v^l - (1 - a)2W - (1 - a)D , 

where W is Gaussian i.i.d. with element variance <7^(A), and D is uniform over Vo and independent 
of W. Note that since -^||D|p = (y'^{h), the resulting mixture also has average per-element variance 
cr2(A). We re-define the loss factor w.r.t. this mixture noise as 

L(A,pe,a) = min|/: Pr|^^ Voj <Pe} • (15) 

Note that this definition reduces to (fT2]) for a = 1. Using this definition, we have the following, which 
is a direct consequence of [20]. 

Proposition 1: (Existence of good lattices) For any error probability pe > 0, and for any < a < 1, 
there exists a sequence of K-dimensional lattices Ak satisfying: 



and 



lim lim L{AK,Pe,Oi) = 1 , (16) 



lim L{Ak) = 1 . (17) 



Note that since by definition, L{AK,Pe,o:) is non-increasing in pe, it follows that for any Pe > this 
sequence of lattices satisfies: 

limsupL(A/f ,Pei a) < 1 • (18) 

In Appendix |I] we elaborate more on the significance of this result, and on its connection to more 
commonly used measures of goodness of lattices. 

III. Modulo-Lattice WZ/DPC Coding 

We now present the joint source/channel scheme for the SI problem of Figure |0l As explained in the 
Introduction, the quadratic-Gaussian rate-distortion function (RDF) of the WZ source © is equal to the 
RDF of the source Qk (without the known part J^), given by: 

2 

Rwz{D) = ^log^ . (19) 
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Figure 0: Analog Wyner-Ziv / dirty-paper coding scheme: S = source, S = reconstruction, Z = channel 
noise, I = interference known at the encoder, J = source component known at the decoder, D = dither 



Similarly, the capacity of the Gaussian DPC © is equal to the AWGN capacity (without the interference 
4): 

C^DPC = ^ log (l + ^) . (20) 

Recalling that the separation principle holds for this problem [4], the optimum distortion ^ is thus given 
by: 

j^opt ^ -J^al . (21) 
P+N ^ ^ ' 

We show how to approach using the joint source/channel coding scheme depicted in Figure |0l 
In this scheme, the K-dimensional encoding and decoding functions ([S]),© are given by: 

X =[/3S + D - al] mod A (22a) 

S =^ I \acY - D - /?J] mod a} + J , (22b) 

respectively, where the second moment ^ of the lattice is cr^(A) = P, and the dither vector D is 

uniformly distributed over Vo and independent of the source and of the channel. The channel power 

constraint is satisfied automatically by the properties of dithered lattice quantization discussed in Section 

im The factors as, ac and /3 will be chosen in the sequel. For optimum performance, /3 which is used 

at the encoder will depend upon the variance of the source unknown part, while ac used at the decoder 

will depend upon the channel SNR. It is assumed, then, that both the encoder and the decoder have full 

knowledge of the source and channel statistics; we will break with this assumption in the next section. 

The following theorem gives the performance of the scheme, in terms of the lattice parameters •, •) 

in ( fTSl ) and in L(-) ([141 ). and the quantities: 

A P , 
"0 =77^' (23a) 

~A / L{A,pe,ao)-l \ 

a = max oq — r — , (J . (23b) 
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We will also use these quantities in the sequel to specify the choice of factors as, etc and (3. 



Theorem 1: (Performance of the MLM scheme with any lattice) For any lattice A and any error 
probability pe > 0, there exists a choice of factors ac,as,/? such that the system of (|22l) (depicted in 
Figure |Oll satisfies: 

where the optimum distortion was defined in (|2TI) . and 

We prove this theorem in the sequel. As a direct corollary from it, taking pe to be an arbitrarily small 
probability and using the properties of good lattices (ITtI ) and ([TSl l. we have the following asymptotic 
optimality resulj^ 

Theorem 2: (Optimality of the MLM scheme) Let D^Kk) be the distortion achievable by the system 
of (l22l ) with a lattice from a sequence {A/^-} that is simultaneously good for source and channel coding 
in the sense of Proposition [T] Then for any e > 0, there exists a choice of factors qc, as and /?, such 
that 

limsupD(Ai^) < D"P* + e . 



For proving Theorem [T] we start with a lemma, showing equivalence in probability to a real-additive 
noise channel (see Figure [Tbl ). The equivalent additive noise is: 

= acZ - (1 - ac)X , (25) 

where Z and X are the physical channel input and AWGN, respectively. By the properties of the dithered 
modulo-lattice operation, the physical channel input X is uniformly distributed over Vo and independent 
of the source. Thus, Zgg is indeed additive and has per-element variance: 

a\ = alN ^{\-acfP • (26) 



'The explicit derivation of ]J^°-^ is not necessary for proving Tfieorem[2j see Appendix III-B I 
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Figure 0: Equivalent channels for the WZ/WDP coding scheme 



Lemma 1: (Equivalent additive noise channel) Fix some pe > 0. In the system defined by ([T]),© 
and (l22l) . the decoder modulo output M (see Figure |0l) satisfies: 

M = PQ + Zeq w.p. (l-pe), (27) 

provided that 

/3V^+<<y7T-^ r , (28) 

where Z^q, defined in (|25] |. is independent of Q and J and has per-element variance a^^ (l26l l. and L{-,-, •) 
was defined in ([TSl l. 

Consequently, as long as ( [28] ) holds, the whole system is equivalent with probability (1 — p^) to the 
channel depicted in Figure [Ibl 



S = J + ^Zeg + asQ 

S + ^Zeg-(l-as)Q . (29) 



Proof: We will first prove equivalence to the channel of Figure [Tal 

M = [(3Q + Zeg] mod A , (30) 
where Zeg was defined in (l25l) . To that end, let T = acY — D — /3J denote the input of the decoder 



modulo operation (see (I22bb and Figure |Oll. Combine Q and (I22al ) to assert: 

T = ac(X + Z + I) -D-/3J 

= [/3S + D - acl] mod A + Z^g + acl - D - /3J . 

Now, using (dJ and the "distributive law" (ITOl ): 

T mod A = [/5Q + Zeg] mod A , 

and since T = M mod A, we establish (l30l ). Now we note that 

PQ + Ze, = /3Q + acZ - (1 - qc)X = ^i-{i-ac?W - (1 - ac)X , 

where W is Gaussian i.i.d., X is uniform over the basic cell Vq of the lattice A, and the total variance 
(per element) is given by the l.h.s. of ( |28l ). By the definition of L(-, •, •), we have that 

T = /3Q + Ze, GVo (31) 

w.p. at least (1 —pe)- Substituting this in (l30b . we get (l27l) . ■ 
This channel equivalence holds for any choice of dimension K, lattice A and factors ac, as and (3, 
as long as ( |28] ) holds. For the proof of Theorem [T] we make the foUowmg choice (using the parameters 
of (US): 

ac =ao , (32a) 

P^=a^ , (32b) 

aP 
aP + aoN 

It will become evident in the sequel, that ac and as are the MMSE (Wiener) coefficients for estimating 
X from X + Z and Q from Q + respectively, while /3 is the maximum zooming factor that allows 
to satisfy (1281 ) with equality, whenever possible. 

Proof of Theorem\l\ For calculating the achievable distortion, first note that by the properties of MMSE 
estimation, 

'^Iq = oicN = aoN . 

Using this, it can be verified that our choice of (3 satisfies ( [28] ). thus ( [291 ) holds with probability (1 — pe). 
Denoting by /^correct j-)incorrect jj^g distortions when ( [29j ) holds or does not hold, respectively, we 
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have: 

■\correct , „ r^incorrect 



^ jjcorrect _j_ ^ jjincorrect (33) 



We shall now bound both conditional distortions. For the first one, we have: 

j^correct ^ ^ ^Z^, - (1 - a5)Q'' 

PK 




(a) (Jeq _ '^Q'^lq 



1 — + a 

= mm{L{A,pe,ac)D"P\al^) <L{A,pe,ac)D''P' , 

where (a) stems from the properties of MMSE estimation. It remains to show that D^^'^o^^^'^* < jjmax^ 
which is established in Appendix Ill-AI □ 

As mentioned in the Introduction, a recent work [15] derives a similar asymptotic result, replacing the 
shaping lattice of our scheme by a random shaping code. Such a choice is less restrictive since it is not 
tied to the properties of good Euclidean lattices, though it leads to higher complexity due to the lack 
of structure. The use of lattices also allows analysis in finite dimension as in Theorem [T] and in Section 
rvl Furthermore, structure is essential in network joint source/channel settings; see e.g. [16]. Lastly, the 
dithered lattice formulation allows to treat any interference signals, see Remark 2 in the sequel. 

We conclude this section by the following remarks, intended to shed more hght on the significance of 
the results above. 

1. Optimal decoding. The decoder we described is not the MMSE estimator of S from Y. This is 
for two reasons: First, the decoder ignores the probability of incorrect lattice decoding. Second, since 
Zeg is not Gaussian, the modulo-lattice operation w.r.t. the lattice Voronoi cells is not equivalent to 
maximum-likelihood estimation of the lattice point (see [20] for a similar discussion in the context of 
channel coding). Consequently, for any finite dimension the decoder can be improved. We shall discuss 
further the issue of working with finite-dimension lattices in Section |Vl 

2. Universality w.r.t. I and J. None of the scheme parameters depend upon the nature of the channel 
interference I and source known part J. Consequently, the scheme is adequate for arbitrary (individual) 
sequences. This has no effect on the asymptotic performance of Theorem |2l but for finite-dimensional 
lattices the scheme may be improved, e.g. if the interference signals are known to be Gaussian with low 
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enough variance. A similar argument also holds when the source or channel statistics is not perfectly 
known, see Section JV] in the sequel. 

3. Non-Gaussian Setting. If the source unknown part Q or the channel noise Z are not Gaussian, the 
optimum quadratic-Gaussian distortion may still be approached using the MLM scheme, though it 
is no longer the optimum performance for the given source and channel. 

4. Asymptotic choice of parameters. In the limiting case where L(A,pe,ao) — ^ 1> we have that 
as = a = ao in (l32l) . i.e. the choice of parameters approaches: 

P 

ac = as = p^ = «o , (34a) 
/3'=«o^ • (34b) 

5. Properties of the equivalent additive-noise channel. With high probability, we have the equivalent 
real-additive noise channel of ( [29l ) and Figure [Tbl This differs from the modulo-additivity of the lattice 
strategies of [20], [21]: Closeness of point under a modulo arithmetic does not mean closeness under 
a difference distortion measure. The condition (1281 ) forms an output-power constraint: No matter what 
the noise level of the channel is, its output must have a power of no more than P; this replaces the 
input-power constraint of the physical channel. Furthermore, by the lattice quantization noise properties 
[18], the "self noise" component (1 — ac)X in (1251 ) is asymptotically Gaussian i.i.d., and consequently so 
is the equivalent noise Zg^. Thus the additive equivalent channel ( |29l ) is asymptotically an output-power 
constrained AWGN channel. 

6. Noise margin. The additivity in ( [29l ) is achieved through leaving a "noise margin". The condition 
(|28] ) means that the sum of the (scaled) unknown source part and equivalent noise should "fit into" the 
lattice cell (see (|3T|)). Consequently, the unknown source part Q is inflated to a power strictly smaller 
than the lattice power P. In the limit of infinite dimension, when the choice of parameters becomes 
(l34l ). this power becomes P'^o'q = a^P. In comparison, it is shown in [21] that in a lattice solution to 
a digital SI problem, if the information-bearing code (fine lattice) occupies a portion of power jP with 
any oq < 7 < 1, capacity is achievecj^ This freedom, however, has to do with the modulo-additivity of 
the equivalent channel; in our joint source/channel setting, necessarily 7 = uq. 

7. Comparison with analog transmission. Lastly, consider the similarity between our asymptotic 
AWGN channel and the optimal analog transmission scheme without SI ([8]): Since we have "eliminated 

''in [22] a similar observation is made, and a code of power aoP is presented as a preferred cfioice, since it allows easy 
iterative decoding between the information-bearing code and the coarse lattice. 
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from the picture" the SI components I and J, we are left with the transmission of the source unknown 
component through an equivalent additive noise channel. As mentioned above, the unknown source part 
Q is only adjusted to power aoP (in the limit of high dimension), while in ^ the source S is adjusted to 
power P; but since the equivalent noise Z(,q has variance a^N, the equivalent channel has signal-to-noise 
ratio of P/N, just as the physical channel. 

IV. Transmission under Uncertainty Conditions 

We now turn to case where either the variance of the channel noise N, or the variance of the source 
unknown part ctq, are unknown at the encodejfl. In Section HV-AI we assume that cjq is known at both 
sides, but the channel SNR is unknown at the encoder. We show that in the limit of high SNR, optimality 
can still be approached. In Section lTV-BI we address the general SNR case, as well as the case of unknown 
ctq; for that, we adopt an alternative broadcast-channel point of view. 

For convenience, we present our results in terms of the channel signal-to-noise ratio 

SNR ^ ^ (35) 



and the achieved signal-to-distortion ratio 



SDR = . (36) 



Denoting the theoretically optimal SDR as SDR''^'*, becomes: 

SDR°P* = 1 + SNR . (37) 

Our achievability results in this section are based upon application of the MLM scheme, generally 
with a sub-optimal choice of parameters due to the uncertainty. We only bring asymptotic results, using 
high-dimensional "good" lattices. We present, then, the following lemma, using the definition: 

Pi = ^ ■ (38) 

Lemma 2: Let SDR(Ai<') be the distortion achievable by the system of ((22)) with a lattice from a 
sequence {A/^} that is good in the sense of Proposition [T] For any choice of factors ac, Q-s and f3, 

liminfSDR(Ai^) > -^-^ , (39) 



^We do not treat uncertainty at the decoder, since can be learnt, while the major insight into the matter of unknown oq 
is gained already by assuming uncertainty at the encoder. 
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provided that 




+ {l-ac f < 1 



(40) 



Proof: This is a direct appUcation of Lemma [T] and of (fTSl ). First we fix some > 0, and note that 
(|40l ) is equivalent to (|28] l. The SDR of the equivalent channel (|29l ). at the limit L(Aft',pe)"c) — > 1 is 
then given by ( [39l ). Then for — ^ the effect of decoding errors vanishes, as shown in Appendix III-BI 

■ 

Note, that by substituting the asymptotically optimal choice of parameters (|34l ) in ( [39l ). the limit 
becomes SDR"p*. 

A. Asymptotic Robustness for Unknown SNR 

Imagine that we know that SNR > SNRq, for some specific SNRq, and that ctq is known. Suppose 
that we set the scheme parameters such that the correct decoding condition (l40b holds for SNRq. Since 
the variance of the equivalent noise can only decrease with the SNR, correct lattice decoding will hold 
for any SNR > SNRq, and we are left with the equivalent additive-noise channel where the resulting 
SDR is a strictly decreasing function of the SNR. We use this observation to derive an asymptotic result, 
showing that for high SNR a single encoder can approach optimality simultaneously for all actual SNR. 
To that end, we replace the choice given in (l32l ). which leads to optimality at one SNR, by the high-SNR 
choice ac = as = 1, where /? is chosen to ensure correct decoding even at the minimal SNRq. 

Theorem 3: (Robustness at high SNR) Let the source and channel be given by ([T|l and (O, respectively. 
Then for any e > 0, there exists an SNR-independent sequence of encoding-decoding schemes (each one 
achieving SDRj^) that satisfies: 



for all sufficiently large (but finite) SNR. I.e., (gB holds for all SNR > SNRo(e), where SNRo(e) is 
finite for all e > 0. 

A limit of a sequence of schemes is needed in the theorem, rather than a single scheme, since for any 
single scheme we have Pf. > 0, thus the effect of incorrect decoding cannot be neglected in the limit 
SNR — > oo (meaning that the convergence in Lemma |2] in not uniform). If we restricted our attention to 
SNRs bounded by some arbitrarily high value, a single scheme would be sufficient. 



liminf SDRx > (1 - e)SDR''P* 



K—*oo 



(41) 



14 









Yi 


DECODER 
1 


Si 


s 


ENCODER 


X 






Ya 


SDRi 
Sa 








DECODER 
2 




BROADCAST 
CHANNEL 


Za tsNRa 




SDRa 



Figure 0: A broadcast presentation of the uncertainty problem. 



Proof: We use a sequence of MLM schemes with good lattices in the sense of Proposition [T] If 
ac = 1, then any 

satisfies the condition (|40l for SNRq, thus for any SNR > SNRq. Here we assume that SNRq > 1, w.l.o.g. 
since we can always choose SNRo(e) of the theorem accordingly. With this choice and with as = 1, we 
have by Lemma |2] that the SDR may approach (for any SNR > SNRq): 

^SNR = 5^!£!L^ . SNR = '"'^o-' . ^1^5- ^ SDR- > 5^55!!^ . SDR-' . 

SNRo SNRo SNR + 1 " SNRq + 1 

Now take e = — 1. Since limsNRn-»oo e = 0, one may find SNRq for any e > as required. ■ 

Note that we have here also a fixed decoder; if we are only interested in a fixed encoder we can adjust 
a5 at the decoder and reduce the margin from optimality. 

B. Joint Source/Channel Broadcasting 

Abandoning the high SNR assumption, we can no longer simultaneously approach the optimal perfor- 
mance (|37]) for multiple SNRs. However, in many cases we can still do better than a separation-based 
scheme. In order to demonstrate that, we choose to alternate our view to a broadcast scenario, where the 
same source needs to be transmitted to multiple decoders, each one with different conditions; yet all the 
decoders share the same channel interference I, see Figure |0l The variation of the source SI component 
J between decoders means that the source has two decompositions: 

S = Qi+Ji = Q2 + J2 , (42) 

and we define the per-element variances of the unknown parts as erf and a 2, respectively. Note that 
this variation does not imply any uncertainty from the point of view of the MLM encoder, as long as 
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SDRj, SDRj, 



(a) SNRi = 2, SNR2 = 10 (b) SNRi = 10, SNR2 = 50 

Figure 0: Broadcast performance. Solid line: Achievable by separation for arbitrary I and J. Dash-dotted 
line: Achievable by MLM for arbitrary I and J. Dashed Une: Achievable by MLM for arbitrary J, with 
1 = 0. Dotted line: Outer bound of ideal matching to both SNRs (achievable by analog transmission 
when I = J = 0). 



af = 0-2', see [23] for a similar observation in the context of source coding. We denote the signal-to- 
noise ratios at the decoders as SNRi < SNR2, and find achievable corresponding signal-to-distortion 
ratio {SDRi,SDR2} pairs. It will become evident from the exposition, that this approach is also good 
for a continuum of possible SNRs. 

We start from the case a\ = (72, for which we have the following. 

Theorem 4: In the broadcast WZ/DPC channel of Figure |0] with erf = o"!, the signal-to-distortions pair 

a -SNRi , a-SNR2 



+ (1 - ac)^SNRi afj + {I - qc)2SNR2 



a = ac[2- ^-^^^«c ] , (43) 



where 



can be approached for any < ac < min ^1, ^j^^j- In addition, if there is no channel interference 
(I = 0), then the pair |l + SNRi, 1 + ^^^^^^^f^^j can be approached as well. 

Proof: As in the proof of Theorem [3j we use Lemma [2] with a choice of (i which allows correct 
decoding in the lower SNR. For the first part of the theorem, fix any ac according to the theorem 
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conditions, and choose any 

p 

where a was defined in (l43l) . in order to satisfy (l40l) . In each decoder, optimize as in (|39l ) to approach 
the desired distortion. For the second part of the theorem, if there is no channel interference, the encoder 
is ac-independent, thus each decoder may work with a different ac value. We can therefore make the 
encoder and the first decoder optimal for SNRi, while the second decoder only suffers from the choice 
of P at the encoder. Again we substitute in ( [39l) to arrive at the desired result ■ 

By standard time-sharing arguments, the achievable SDR regions include the convex hull (in the 
distortions plane) defined by these points and the trivial {1 + SNRi, 1} and {1, 1 + SNR2} points. Figure 
|0] demonstrates these regions, compared to the ideal (unachievable) region of simultaneous optimality 
for both SNRs, and the separation-based region achieved by the concatenation of successive-refinement 
source code (see e.g. [24]) with broadcast channel code [25] (about the sub-optimality of this combination 
without SI, see e.g. [26]). It is evident, that in most cases the use of the MLM scheme significantly 
improves the SDR tradeoff over the performance offered by the separation principle, and that the scheme 
approaches simultaneous optimality where both SNRs are high, as promised by Theorem [3] Note that, 
unlike the separation-based approach, the MLM approach also offers reasonable SDRs for intermediate 
SNRs. Moreover, note that this region is achievable when no assumption is made about the statistics of 
I and J. If these interferences are not very strong comparing to P and Uq, respectively, then one may 
further extend the achievable region by allowing some residual interference. 

To conclude, we briefly discuss the case where erf / (t|. We define the SDR of each decoder relative 
to its own variance, and ask what are the achievable SDRs for a pair of SNRs, which may be equal or 
different. Assume here the simple case, where there is no channel interference, i.e. I = 0. In this case, 
the encoder only needs to agree upon /3 with the decoders, thus (by Lemma we may approach for 
n = 1,2: 

SDR„ = 1 + -^SNR„ , (44) 

Popt,n 

where Popt.n is the optimum choice of /? for SNR„ according to ( |34l ). It follows, that if the two decoders 
require the same value of f3, they may be both approach the theoretically optimal distortion. This translates 
to the optimality condition: 

2 1 + SNRi 2 1 + SNR2 

fJ 1 = (To . 

^ SNRi SNR2 

This scenario was presented in [27], where simultaneous optimality using hybrid digital/analog schemes 
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was proven under a different condition: 

SNRi SNR2 

Both conditions reflect the fact that better source conditions (lower cTq) can compensate for worse channel 
conditions (lower SNR). It follows from the difference between the conditions, that for some parameter 
values the MLM scheme outperforms the approach of [27], thus extending the achievable SDRs region. 

V. Discussion: Delay and Complexity 

We have presented the joint source/channel MLM scheme, proven its optimality for joint WZ/DPC 
setting with known SNR and shown its improved robustness over a separation-based scheme. We now 
discuss the potential complexity and delay advantages of our approach relative to separation-based 
schemes, first considering the complexity at high dimension and then suggesting a scalar variant. 

Consider a separation-based solution, with source and channel encoder/decoder pairs. An optimal 
channel coding scheme typically consists of two codes: an information-bearing code and a shaping code, 
both of which require a nearest-neighbor search at the decoder. An optimal source coding scheme also 
consists of both a quantization code and a shaping code in order to achieve the full vector quantization 
gain (see e.g. [28]), thus two nearest-neighbor searches are needed at the encoder. The MLM approach 
omits the information-bearing channel code and the quantization code, and merges the channel and source 
shaping codes into one. It is convenient to compare this approach with the nested lattices approach to 
channel and source coding with SI [10], since in that approach both the channel and source information 
bearing/shaping code pairs are materialized by nested lattices. In comparison, our scheme require only 
a single lattice (parallel to the coarse lattice of nested schemes), and in addition the source and channel 
lattices collapse into a single one. 

There is a price to pay, however: For the WZ problem, the coarse lattice should be good for channel 
coding, while for the WDP problem the coarse lattice should be good for source coding [10]. The lattice 
used for MLM needs to be simultaneously good for source and channel coding (see Appendix lU). While 
the existence of such lattices in the high dimension limit is assured by [19], in finite dimension the 
lattice that is best in one sense is not necessarily best in the other sense [29], resulting in a larger 
implementation loss. Quantitively, whereas for source coding the lattice should have a low normalized 
second moment, and for channel coding it should have a low volume-to-noise ratio, for joint source 
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Figure 0: Scalar MLM/companding scheme for joint source/channel coding over a high-SNR dirty-paper 
channel: S - source, S - reconstruction, Z - channel noise, / = interference known at the encoder, 
g{-) - companding function. 



channel coding the product L{A,pe) (fT2l) should be lowo (see Appendix IH). The study of such lattices 
is currently under research. Exact comparison of schemes in high dimension will involve studying the 
achieved joint source/channel excess distortion exponent (see [30] for a recent work about this exponent 
in the Gaussian setting). 

From the practical point of view, the question of a low-dimensional scheme is very important, since 
it implies both low complexity and low delay. One may ask then, what can be achieved using low- 
dimensional lattices, e.g. a scalar lattice? The difficulty, however, is that in low dimensions a low 
probability of incorrect decoding Pf. implies a high loss factor L{K,pe), thus the distortion promised 
by Theorem [T] grows. Some improvement may be achieved by using an optimal decoder rather than the 
one described in this work (see Remark 1 at the end of Section Hill) , an issue which is left for further 
research. A recent work [31] suggests an alternative, for the case of channel interference only (J = 0), 
by also changing the encoder: The scalar zooming factor /3 of the MLM scheme is replaced by non-linear 
companding of the signal; see Figure |0l At high SNR, the distortion loss of such a scalar MLM scheme 
with optimal companding comparing to ^ is shown to be 

jjcompanding y^vr 

— — = = 4.3dB . 

£)Opt 2 

In comparison, the loss of a separation-based scalar scheme, consisting of a scalar quantizer and a scalar 
(uncoded) channel constellation, is unbounded in the limit SNR oo. This is since in a separation-based 
scheme the mapping of quantized source values to channel inputs is arbitrary; consequently, keeping the 

^In Theorem [T] we show that the figure of merit is L{A,pe, a) il5i . but for reasonably high SNR it seems that the effect of 
self noise should not be too dominant, so we can set q = 1. 
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loss bounded implies that the error probability must go to zero in the high-SNR limit, and the gap of a 
scalar constellation from capacity grows. 
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Appendix I 
Measures of Goodness of Lattices 

In this appendix we discuss measures of goodness of lattices for source and channel coding, and their 
connection with the loss factor relevant to our joint source/channel scheme. 

When a lattice is used as a quantization codebook in the quadratic Gaussian setting, the figure of merit 
is the lattice normalized second moment: 

0(A) ^ , (45, 

where the cell volume is V{K) = /^^ dx. By the isoperimetric inequality, G(A) > where is the 
normahzed second moment of a ball with the same dimension K as the lattice. This quantity satisfies 
— 2^' ^^^^ asymptotic equaUty in the limit of large dimension. A sequence of K-dimensional 
lattices is said to be good for MSE quantization if 

lim G{Ak) = ^ , (46) 

thus it asymptotically achieves the minimum possible lattice second moment for a given volume. 

When a lattice is used as an AWGN channel codebook, the figure of merit is the lattice volume-to-noise 
ratio at a given error probability 1 > Pe > (see e.g. [32], [20]): 

l,{A,p,)^}^^ , (47) 
where a\ is the maximum variance (per element) of a white Gaussian vector Z having an error probabiUty 

Pr{Z^Vo}<Pe ■ 

For any lattice, /i(A,pe) > /^^(Pe), where /i^(pe) is the volume-to-noise ratio of a ball with the same 
dimension K as the lattice. For any 1 > > 0, fi*j^{pe) > 27re, with asymptotic equality in the limit of 
large dimension. A sequence of K-dimensional lattices is good for AWGN channel coding if 

lim lim /i(Ax,Pe) = 27re , (48) 
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thus it possesses the property of having a minimum possible cell volume such that the probability of an 
i.i.d. Gaussian vector of a given power to fall outside the cell vanishes. 

Combining the definitions (l45l ) and (l47l ). we see that the loss factor L(A,pe) (O satisfies: 

L{k,pe) = G{K) ■ ^l{k,pe) . 

Furthermore, the existence of a good sequence of lattices in the sense of (fT3l) is assured by the existence 
of a sequence that simultaneously satisfies (l46l) and (l48l) . which was shown in [19, Theorem 5]. 

Proposition [T] is implicit in the proof of [20, Theorem 5]. It is based upon the existence of lattices that 
are simultaneously good for AWGN channel coding and for covering [19], where goodness for covering 
also implies goodness for MSB quantization; for such lattices, it is shown that the mixture noise cannot 
be much worse than a Gaussian noise of the same variance. Later, it was shown in [33] that, for such 
lattices, for small enough error probability p^, the introduction of self noise actually reduces the loss 
factor, i.e. L(A,pe,a) < L{K,Pf.,\). 

Appendix II 

The effect of Decoding Failure on the Distortion 



With probability pe, correct lattice decoding fails, i.e. ( [311 ) does not hold. These events contribute to 
the total distortion a portion of 

I) = • D^™^^* , (49) 

where 2)«"corT-erf ^j^g distortion given a decoding failure, as in the proof of Theorem [T] In this Appendix 
we quantify this effect: In the first part we show that D"^"-^ of (l24l) is a (rather loose) bound on u^^'^o^rect ^ 
thus completing the proof of Theorem [T] In the second part, we show directly that D must vanish in the 
limit of small p^, without resorting to an explicit bound on D^^^^orrect^ 
In both parts we use the observation that 



S - S = Q - Q , (50) 

_rflr»_L'7. 1 rr^r^A A 



where Q = ^[/^Q + Ze,] mod A, see also Figure [Tbl We note that although Q is unbounded, we always 



have that 



QGyVo . (51) 
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A. A Bound on the Conditional Distortion for Any Lattice 

In order to complete the proof of Theorem [H we now bound D^"-<^o'^^f^<^t of (|33 



j^incorrect ^ —E{\\S - Sf \(3Q + Z^g ^ Vq} 

i?{||Q-Qf|/3Q + Ze, ^ Vo} 
< ^(i?{||Qf|/3Q + Ze, ^ Vo} + i^{||Qf|/3Q + Ze, ^ Vol) , (52) 



1 

K' 

— FJWCl - 0|i2 
K 

2 

K 

where the inequaUty follows from assuming maximizing (—1) correlation coefficient and then applying 
the Cauchy-Schwartz inequality. We shall now bound these two terms. For the first one, recalling the 
definition of the covering radius (fTTI) . we bound the conditional expectation by the maximum possible 
value: 

E{\\Qf\PQ + Ze, ^ Vol < maxdIQf ) = ^ J ' < -Li . (53) 



For the second term, we have: 

E{||Qf|/?Q + Zeg^ Vol < E{\\Qf\(3Q^Vo} 

< E{\\Qf\f3Q ^ Bo} , 
where Bq is the circumsphere of Vo, of radius r(A). It follows that 

E{\\Qf\PQ + Ze, ^ Vol < alE{V\V > vq} , 

9 A r^(A) 

where V ~ and vq = ^ a ■ This conditional expectation is given by: 



Q(f + l,T; of) 



E{V\V > vo} = ^'Zk kV ^ ^0 + 2 



where Q(-, •) is the regularized incomplete Gamma function, and the inequality can be shown by means 
of calculus. This gives the bound on the second term: 

i?{||Qf I/3Q + Z,, i Vo} < + 2Kal 



Substituting this and (1531 ) in (I52b . we have that: 



T^incorrect ^ a 



Recalling the choice of /? in (I32bl ) and the definition of L(-, •) in ([T4l ). the bound follows. 
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B. Asymptotic Effect of Decoding Failures 

In this part we follow the claims used by Wyner in the source coding context to establish [5, (5.2)], 
to see that lirrip^^o D = 0, where D was defined in ( |49l) . without using the explicit bound derived in 
Appendix III-AI This serves as a simpler proof of Theorem |2l moreover, it also applies to a non-optimal 
choice of parameters, thus it serves in the analysis of performance under uncertainty conditions. 

Denoting the decoding failure event by e and its indicator by /g, and recalUng (l50l) . we re-write the 
contribution to the distortion as: 

D = E{I,. iQ-Qf} . 

For any value of the source unknown part Q, the distortion is bounded by: 

d(Q) = sup(Q - Q)2 . 
Q 

The expectation E{d{Q)} is finite, since Q is Gaussian and Q is bounded (see (ISTIl). We now have that 

D<E{Ie-d{Q)} . 

Using a simple lemma of Probability Theory [5, Lemma 5.1], since E{d{Q)} is finite, this expectation 
approaches zero as p{e) = pe ^ 0. 
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